Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric IRT
نویسندگان
چکیده
In recent years, as cognitive theories of learning and instruction have become richer, and computational methods to support assessment have become more powerful, there has been increasing pressure to make assessments truly criterion referenced, that is, to “report” on student achievement relative to theory-driven lists of examinee skills, beliefs and other cognitive features needed to perform tasks in a particular assessment domain. Cognitive assessment models must generally deal with a more complex goal than linearly ordering examinees, or partially ordering them in a low-dimensional Euclidean space, which is what item response theory (IRT) has been designed and optimized to do. In this paper we consider some usability and interpretability issues for single-strategy cognitive assessment models that posit a stochastic conjunctive relationship between a set of cognitive attributes to be assessed, and performance on particular items or tasks in the assessment. The attributes are coded as present or absent in each examinee, and the tasks are coded as performed correctly or incorrectly. The models we consider make few assumptions about the relationship between latent attributes and task performance beyond a simple conjunctive structure: all attributes relevant to task performance must be present to maximize probability of correct performance of the task. We show by example that these models can be sensitive to cognitive attributes even in data that was designed to be well-fit by the Rasch model, and we consider several stochastic ordering and monotonicity properties that enhance the interpretability of the models. We also identify some simple data summaries that are informative about the presence or absence of cognitive attributes, when the full computational power needed to estimate the models is not available. Cognitive Assessment and NPIRT 1
منابع مشابه
Cognitive Assessment Models With Few Assumptions, and Connections With Nonparametric Item Response Theory
متن کامل
Some topics in nonparametric and parametric IRT, with some thoughts about the future
1 Abstract Item response theory (IRT) has arguably been one of the most successful and widely used statistical modeling techniques in psychometrics, with applications in developmental, social, educational and cognitive psychology for example, as well as in medical research, demography and other social science settings. In this paper I will try to briefly summarize some of the important research...
متن کاملKernSmoothIRT: An R Package allowing for Kernel Smoothing in Item Response Theory
Item Response Theory (IRT) models enable researchers to evaluate test or survey subjects and questions simultaneously to more accurately judge the difficulty and quality of the test as well as the strength of each subject. Most IRT analyses use parametric models, often without satisfying the necessary assumptions of these models. The KernSmoothIRT package uses kernel smoothing from Ramsay (1991...
متن کاملAnalyzing psychopathology items: a case for nonparametric item response theory modeling.
The authors discuss the applicability of nonparametric item response theory (IRT) models to the construction and psychometric analysis of personality and psychopathology scales, and they contrast these models with parametric IRT models. They describe the fit of nonparametric IRT to the Depression content scale of the Minnesota Multiphasic Personality Inventory--2 (J. N. Butcher, W. G. Dahlstrom...
متن کاملFitting Item Response Theory Models to Two Personality Inventories: Issues and Insights.
The present study compared the fit of several IRT models to two personality assessment instruments. Data from 13,059 individuals responding to the US-English version of the Fifth Edition of the Sixteen Personality Factor Questionnaire (16PF) and 1,770 individuals responding to Goldberg's 50 item Big Five Personality measure were analyzed. Various issues pertaining to the fit of the IRT models t...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2001